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Theory  Reform  Caused  by  an  Argumentation  Tool 


Kurt  VanLehn 

Abstract 

Recently,  software  aids  have  been  developed  for  formulating  and 
managing  arguments.  Most  of  these  programs  are  combinations  of  text 
editors  and  databases.  This  paper  concerns  one  such  system,  called 
NoteCards.  The  paper  discusses  two  incidents  where  using  the  system 
uncovered  major  flaws  in  the  arguments.  These  discoveries  were  quite 
unexpected.  This  paper  discusses  these  two  incidents  and  trysto 
ascertain  why  a  such  a  simple  tool  had  such  a  profound  impact  and  what 
the  tool's  future  might  be. 


Theory  reformulation  caused  by  an  argumentation  tool 


Knit  VanLehn 


There  are  many  projects  whose  goal  is  to  produce  a  network  of  well-reasoned  arguments  in  support  of 
some  assertions.  Kxamples  of  such  networks  are  a  legal  brief,  a  market  analysis  or  a  scientific  theory. 
Until  recently,  computer  technology  for  developing  such  networks  was  concerned  mostly  with  the 
acquisition  of  information  to  feed  into  die  arguments.  Information  retrieval  systems  help  the  lawyer 
find  precedents.  Survey  instruments  and  statistical  packages  help  the  market  analyst  quantify  market 
forces.  Computer-driven  instruments  line  the  benches  of  the  physical  scientist's  laboratory.  Such  tools 
help  find  die  facts  upon  which  the  arguments  rest,  but  they  don't  help  the  rcasoncr  invent,  record, 
manage  or  modify  die  arguments  themselves.  For  manipulating  arguments,  dicse  professionals  have 
had  to  rely  on  paper  technologies,  such  as  index  cards  and  file  folders,  or  their  electronic  analogs,  text 
editors  and  file  systems. 

Recently,  software  aids  have  been  developed  for  formulating  and  managing  arguments.  Most  of  these 
programs  are  combinations  of  text  editors  and  databases.  This  paper  concerns  one  such  system,  called 
NoteCards.  which  is  being  developed  at  Xerox's  Intelligent  Systems  1  aboratory.  The  paper  reports  the 
authors's  experience  in  using  the  system.  It  discusses  two  incidents  where  using  the  system  uncovered 
major  flaws  in  the  arguments.  These  discoveries  were  quite  unexpected.  The  expectation  had  been  that 
NoteCards  would  make  argumentation  easier,  but  not  that  it  would  change  its  quality. 

T  his  paper  discusses  these  two  incidents  and  trys  to  ascertain  why  a  such  a  simple  tool  had  such  a 
profound  impact  and  what  the  tool's  fijturc  might  be.  The  discussion  and  analysis  is  necessarily  informal 
and  even  anecdotal  at  times.  Such  a  treatment  cannot,  of  course,  substitute  for  careful  experimentation. 
However,  it  seems  worth  reporting  these  incidents  now  because  they  raise  certain  interesting, 
non-obvious  issues.  This  informal  presentation  of  the  issues  may  lay  the  groundwork  for  more  formal 
studies. 

The  two  incidents  occurred  a  month  apart.  The  second  one  is  simpler  to  describe,  so  it  will  be  presented 
first,  before  that,  a  little  background  on  my  project  is  ncccsary. 


Background 

The  project  is  to  develop  a  psychological  theory  of  how  people  learn  procedures,  such  as  arithmetic 
procedures,  clerical  procedures,  or  procedures  for  analyzing  electronic  circuits  (see  Van  I  elm,  1983a.  foi 
a  synopsis).  The  main  data  arc  a  large  collection  of  systematic  errors  (called  bugs)  that  were  observed  in 
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the  behavior  of  114’  elementary  school  students  who  were  learning  arithmetic.  Most  bugs  can  be 
explained  more  than  one  wav  depending  on  what  hypotheses  one  makes  about  the  learning  process, 
lhe  main  job  of  argumentation  is  to  contrast  the  explanatory  power  of  various  sets  of  hypotheses. 
Metaphorically  speaking,  the  theorist  takes  several  sets  of  hypotheses  and  sees  how  many  hugs  each  can 
explain. 

In  practice,  live  argumentation  docs  not  contrast  multiple  sets  of  hypotheses.  Instead,  a  divide  and 
conquer  strategy  is  used.  I  he  overall  question  —  how  arc  procedures  learned?  —  is  divided  into  a  set  of 
:ssucs.  Hacli  issue  has  several  alternative  hypotheses  to  explain  it.  One  suns  with  very  general  issues, 
such  as  "Do  students  learn  from  worked  example  exercises,  or  from  analogies  to  familiar  procedures,  or 
from  written  descriptions  of  die  procedure,  or  what?"  Depending  on  which  hypothesis  wins  the 
argument,  more  specific  issues  arc  formulated,  c.g..  "Given  that  students  learn  procedures  from  worked 
examples,  how  do  they  learn  conditional  branches?"  (A  conditional  branch  is  a  chunk  of  procedure  of 
the  form  If  such-and-such  then  do  X  else  do  Y.)  lhe  relationship  between  issues,  hypotheses  and 
subissues  will  be  discussed  later  in  detail.  The  point  here  is  only  that  instead  of  formulating  a  huge 
argument  that  contrasts  whole  sols  of  hypotheses,  one  considers  a  set  of  issues  and  for  each  issue, 
formulates  an  argument  contrasting  several  indiv  idual  hypotheses. 

The  two  incidents  of  NoteCard-induccd  theory  reformulation  are  complementary.  One  incident 
concerns  the  structure  of  uimr-issuc  reasoning,  and  the  other  incident  concerns  the  structure  of 
mterh%uc  reasoning.  1  o  put  it  differently,  the  first  incident  concerns  the  structure  of  an  argument  about 
a  single  issue,  i.c.,  how-  to  compare  several  hypotheses  and  choose  a  winner.  The  second  incident 
concerns  representing  die  relationships  between  issues. 

Intra-issue  argumentation 

NotcCards  deliberately  docs  not  force  an  argument  structure  on  the  user.  In  this  respect,  it  differs  from 
lhmk  lank  and  other  outlining  tools  which  force  the  user  to  employ  a  tree  structure.  This  extra 
flexibility  allows  users  to  use  multiple  argument  structures  and  to  change  their  arguments  from  one 
structure  to  another.  The  incident  to  be  described  in  this  section  was  precipitated  when  the  theory’s 
arguments  were  converted  from  one  format  (a  tree  structure)  to  another  (a  matrix  structure). 

The  NotcCards  system  is  based  on  a  simple  idea:  an  electronic  3-by-5  card.  The  database  is  a  set  of 
"notccards."  Figure  1  shows  several  notccards  as  the  user  normally  sees  them  on  the  screen.  A 
notecard  has  a  title,  w  hich  shows  in  the  dark  bar  on  die  top  of  the  card.  The  body  of  a  notccard  is  text 
and/or  graphics.  All  the  notccards  shown  in  figure  1  have  textual  bodies.  A  notccard's  body  may 
contain  pointers.  In  the  middle  card  of  figure  1.  there  arc  four  pointers.  Pointers  have  labels,  l  he  label 
of  the  fust  pointer  is  "Remark.”  and  the  label  of  the  second  pointer  is  "Rebuttal."  l  he  other  two 
pomtc’s  are  labelled  as  well,  but  the  user  has  chosen  a  display  format  for  these  pointers  that  docs  not 
snow  ihe  labels.  I  here  are  no  contraints  on  the  vocabulary  of  pointer  labels:  the  user  may  create  new 
iaMs  a  w  ill  I  lie  destination  of  a  pointer  is  another  card,  l  he  destination  of  the  first  pointer  in  the 
iroJ.I'e  .mi  is  the  top  card.  I  he  destination  of  the  other  three  pointers  is  the  bottom  card.  There  arc  no 
unison  ihe  topologv  of  pointers.  A  card  can  point  to  itself  or  any  other  card. 


Insert  figure  1  about  here. 


Inch  card  is  an  instance  of  a  WYSIWYG  editor  (what  you  see  is  what  you  get)  that  uses  a  mouse  as  a 
pointing  device.  When  the  user  points  at  a  pointer  and  clicks  the  mouse  button,  the  card  that  is  the 
destination  of  the  pointer  is  fetched  from  the  notecard  database  and  displayed  on  the  screen.  This  allows 
one  to  flip  quickie  through  cards.  Mane  other  facilities  are  provided  by  MoieCards.  but  the  preceding 
brief  introduction  will  suffice  for  now. 

As  mentioned  earlier,  a  change  in  tire  format  of  arguments  led  to  uncoeering  flaws  in  die  theory.  The 
old  format  was  a  tree.  Figure  2  displays  such  a  tree  as  a  schematic  outline.  This  tree  structure  reflects  a 
standard  rhetorical  structure.  The  format  has  die  advantage  that  breadth-first  traversal  of  the  tree  is  an 
expositional  sequence  that  makes  sense.  In  fact,  the  original  NotcCards  database  was  constructed  from  a 
32S-pagc  document  (Yanl.ehn.  1983b)  whose  expositional  structure  was  a  breadth-first  traversal  of  die 
argument  trees.  That  is.  each  chapter  was  a  discussion  of  a  single  issue.  A  chapter  began  with  a 
statement  of  the  issue  and  a  list  of  the  competing  hypotheses.  Fach  of  the  remaining  sections  of  the 
chapter  discussed  a  single  hypothesis,  giving  the  arguments  for  and  against  it.  I  he  document  had  about 
20  chapters,  with  three  to  seven  hypotheses  per  chapter.  Because  the  document  was  carefully  structured 
and  die  NotcCards  format  was  chosen  to  reflect  diis  structure,  it  was  easy  to  convert  the  document  into  a 
NotcCards  database.  It  took  only  ten  days. 


Insert  figure  2  about  here. 


The  incident  occurred  while  new  hypotheses  were  being  added  to  the  theory.  A  certain  issue  already 
had  four  hypotheses.  Three  more  hypotheses  had  been  invented.  Their  empirical  implications  looked 
promising,  although  a  little  confusing.  Putting  the  new  hypotheses  into  the  NotcCards  system  should 
have  clarified  which  of  the  seven  hypotheses  was  the  best.  However,  die  resulting  tree  structure  did  not 
make  it  much  easier  to  tell  which  hypothesis  was  the  winner.  The  actual  empirical  facts  (i.c.,  bugs) 
dangled  off  the  leaves  of  a  bushy  tree  of  notccards.  A  summary  card  was  needed  that  would  show  the 
hypotheses  and  facts  in  a  compact  way.  For  each  fact  and  each  hypothesis,  it  should  show  whether  the 
fact  supported  the  hypothesis  or  not.  The  obvious  organization  was  a  Cartesian  product  (sec  Figure  3). 


Insert  figure  3  about  here. 


When  the  issue  card  mentioned  above  was  converted  to  this  matrix  format,  each  of  the  seven  hypotheses 
had  its  own  row.  t  here  were  six  columns,  one  for  each  of  the  facts.  However,  not  all  the  cells  of  the 
matrix  were  filled  in.  I  had  neglected  to  evaluate  some  of  i'e  hypotheses  against  some  of  the  facts.  Ihe 
('Id  tree  structure  made  it  easy  to  overlook  such  mistakes.  The  matrix  format  made  them  stick  out  as 
blank  cells  in  the  matrix.  Not  much  research  was  needed  to  provide  the  needed  evaluations  and  fill  in 
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the  Wank  cells,  and  set.  the  results  were  quite  surprising.  All  three  of  the  new  hypotheses  turned  out 
poorly .  Of  the  four  old  hypotheses,  the  one  that  was  thought  best  actually  turned  out  quite  badly.  A 
previously  rejected  hypothesis  turned  out  best.  In  short,  sloppy  reasoning,  abetted  by  a  poor  rhetorical 
organization,  allowed  the  suppression  of  a  winning  hypothesis.  The  new  matrix  organization  uncovered 
the  mistakes,  leading  to  an  improved  theory. 

I  his  incident  was  not  unique.  Several  reversals  of  the  same  kind  occurred  as  die  rest  of  the  arguments 
were  converted  to  matrix  format.  That  these  errors  could  remain  undetected  for  several  years  is  even 
more  amazing  when  one  considers  how  many  people  had  read  or  listened  to  the  arguments.  The 
implications  of  this  incident  will  be  discussed  after  the  second  incident  has  been  discribcd. 

Inter-issue  argumentation 

\  common  device  in  the  argumentation  is  to  use  an  independently  motivated  hypothesis  to  help  defend 
one  of  the  hypotheses  under  discussion.  For  instance,  if  A  and  B  arc  competing  hypotheses,  and  X  is  the 
winning  hypothesis  of  some  other  issue's  competition,  one  argues  "X  and  A  together  predict  F.  whereas 
X  and  B  predict  not-F.  Since  F  is  empirically  true,  A  is  a  better  hypothesis  than  B.”  The  argument  for  A 
assumes  X.  This  is  one  kind  of  inter-argument  relationship,  the  most  common  one.  This  intcr-issue 
relationship  was  represented  in  NotcCards  by  having  the  argument  cards  for  A  and  B  have  a  pointer  to 
X  labelled  "premise.” 

Explicitly  representing  intcr-issuc  relationships  was  a  major  reason  for  converting  tlic  document  to 
NotcCards.  As  the  theory  changed,  it  was  important  to  know  which  arguments  depended  on  which 
hypotheses.  For  instance,  if  hypothesis  X  is  refuted  by  new  evidence,  then  one  must  retract  the 
arguments  that  have  X  as  a  premise.  This  was  difficult  given  just  the  document.  Although  one  could 
open  the  document  to  the  section  that  describes  hypothesis  X  in  order  to  see  the  arguments  for  and 
against  it.  the  document  was  not  cross-indexed  in  such  a  way  that  one  could  find  all  the  arguments  that 
depended  on  X.  NotcCards  automatically  provides  such  a  cross-index.  Attached  to  the  card  for  X  is  a 
list  of  all  the  pointers  that  point  to  X.  This  cross-index  makes  it  easier  to  revise  the  theory. 

After  a  few  weeks  of  theory  revision,  it  seemed  likely  that  these  inter-issue  pointers  could  provide  a  nice 
geometric  summary  of  the  theory  as  a  whole.  NotcCards  can  automatically  construct  layout,  and 
display  a  directed  graph  whose  nodes  are  card  titles  and  whose  links  correspond  to  pointers  running 
between  a  pair  of  cards.  This  facility  is  called  a  browser.  Similar  facilities  in  programming  environments 
have  been  found  to  be  extremely  useful  in  summarizing  complex  programs.  It  seemed  likely  that 
browsing  the  NotcCards  database  using  the  premise  pointers  would  yield  a  helpful  overview  of  the 
theory. 

When  an  attempt  was  made  to  browse  the  database,  an  unexpected  propertv  of  the  argumentation  was 
discovered:  the  graph  created  bv  the  biowser  was  disconnected.  There  were  sets  of  issues  th.it  were 
totally  unrelated  to  oilier  sets.  Intuitively,  tins  shouldn't  be  so.  Because  the  issues  are  all  part  of  the 
same  theory,  tlicv  must  relate  somehow  I  he  browser  revealed  that  some  essential  intcr-issuc 
'claton-.hips  had  not  been  made  explicit. 


I  Ins  lunched  off  a  examination  of  the  epistemology  of  inter- issue  relationships.  It  was  discovered  that 
important  assumptions  had  been  made  without  mentioning  them  anywhere.  The  assumptions  were  all 
of  a  certain  kind:  a  large  issue  was  decomposed  into  smaller  issues.  For  example,  the  large  issue  "what  is 
the  student's  mental  representation  of  die  skill  they  are  learning."  was  decomposed  into  three  smaller 
issues: 

1.  What  is  die  representation  of  perceptual  knowledge  about  the  skill's  environment,  c.g.. 

►  lor  the  subtraction  skill,  a  grammar  for  the  multicolumn  notation. 

►  For  algebra  equation  sols  mg.  a  grammar  for  algebraic  equations. 

2.  W  hat  is  die  representation  of  procedural  knowledge,  c.g., 

►  For  subtraction,  a  procedure  for  writing  and  scratching  out  digits. 

►  For  algebra  a  procedure  for  selecting  algebraic  transformations. 

3.  \\  hat  is  the  representation  of  factual  know  ledge,  c.g., 

►  For  subtraction,  a  table  of  number  facts  such  as  7-5  =  2. 

►  For  algebra,  a  table  of  facts  such  as  "-  is  the  opposite  sign  of  +’*. 

The  division  of  task-specific  knowledge  into  notational.  procedural  and  factual  knowledge  was  never 
mentioned  explicits  in  die  document.  There  were  several  such  decompositions,  none  of  which  had  been 
rccogni/ed  in  the  document.  The  NoteCards  system  browser  revealed  that  these  decompositions  were 
needed  in  order  to  complete  the  argumentation. 

The  obvious  cure  was  to  add  a  new  type  of  notccard.  called  a  decomposition,  to  describe  the  division. 
When  this  was  done,  it  became  clear  that  these  decompositions  were  doing  almost  as  much  'work'  is  the 
hypodicses.  At  that  time,  there  were  36  issues;  seven  were  handled  by  decomposition  and  29  were 
handled  by  competitive  argumentation  among  hypodicses.  Clearly,  these  decompositions  needed  to  be 
subjected  to  closer  scrutincy.  I  he  best  way  would  be  to  hold  competitions  among  alternative 
decompositions.  It  didn't  take  long  to  figure  out  alternative  decompositions  to  the  seven  original  ones 
(which  is  significant  in  itself). 

One  of  the  new,  alternative  decompositions  turns  out  to  have  very  interesting  properties.  It  is  an 
alternative  to  the  decomposition  mentioned  above.  This  dccompostion  is  called  the  annotated  grammars 
decomposition  because  it  replaces  two  knowledge  representations  —  the  notational  grammar  and  the 
procedure,  which  are  parts  1  and  2  in  die  list  above  —  with  a  unified  representation  called  an  annotated 
grammar.  Whereas  grammars,  procedures  and  tables  of  facts  arc  common  representations  in  computer 
scincc.  annotated  grammars  are  new.  Although  it  is  not  worth  explaining  the  tcchical  details  in  this 
paper,  it  seems  now  that  annotated  grammars  are  a  much  better  model  of  student's  knowledge 
structures,  even  lor  the  supposedly  "procedural"  skills  of  arithmetic!  The  annotated  grammars 
decomposition  is  a  major,  and  welcome,  revision  to  the  theory. 

I  he  discovery  of  the  annotated  grammais  decomposition  was  provoked  by  an  observed  incompleteness 
in  the  aigumcm  structure  of  the  theory.  NoteCards  made  observing  this  incompleteness  easy,  and 
perhaps  even  inevitable  H,  the  wav.  once  the  seven  new  decomposition  issues  were  added  to  provide  a 
home  for  the  new  dec- imposition  competitions  the  browser  created  a  connected  graph  (see  figure  4) 


I  ho  odes  with  ,i  "<"  suffix  label  the  decomposition  issues.  Without  them,  the  graph  would  be  quite 

diM.o;ip.se’.ed. 

Inscit  figure'  4  about  here. 


Discussion 

IHpcr  note  cards  luxe  been  used  for  decades  for  organizing  arguments,  although  I  did  not  happen  to  use 
them  u:  developing  mx  theory.  Although  NotcCards  may  be  faster  and  more  conxenicnt  titan  paper 
note  cards,  but  is  there  any  reason  to  believe  that  NotcCards  encourage  better  quality  argumentation 
than  pape:  note  cards?  1  hat  is.  would  the  incidents  just  described  luxe  happened  if  1  had  used  paper 
note  cards  instead  «>f  NotcCards'.’  1  here  arc  sox  oral  reasons  to  doubt  that  they  would  have,  l-'irst, 
hex  uisc  paper  note  cards  are  three-dimensional  objects,  the  easiest  way  to  implement  scratch 
organizations  of  paper  note  cards  is  to  arrange  them  spatially.  Hoxxcxer.  when  there  are  more  than 
about  100  note  cards,  this  gets  cumbersome.  (Incidently.  there  are  about  800  NotcCards  in  my 
database.)  When  there  are  many  paper  note  cards,  it  is  inevitable  that  they  are  arranged  spatially  along 
their  thi nest  dimension:  they  are  filed  in  a  file  box.  The  most  complex  topology  that  can  fit  in  a  single 
dimension  is  a  tree.  Paper  note  cards  encourage  dcndralic  organizations,  unless  the  person  is  working 
w.th  a  small  number  of  cards.  In  short,  when  one  is  fooling  around  with  various  scratch  organizations, 
nape:  note  cards  don't  encourage  one  to  search  the  whole  space  of  possible  organizations.  To  put  it 
more  parochially.  I  doubt  that  I  would  have  tried  a  matrix  format  for  arguments  if  I  had  been  using 
paper  note  cards.  Consequently,  die  theory  would  still  contain  many  flawed  arguments. 

!  acre  is  no  doubt  that  NotcCards  encourages  one  to  "fool  around  with  scratch  organizations."  (s  this 
ei'.  d’  I  will  argue  that  it  is.  by  using  an  analogy  to  text  editing.  One  often-heard  complaint  about 
modem  text  editors  .s  that  they  encourage  writers  to  fool  around  with  the  format  of  the  text,  and  thus 
the  document  preparation  process  down.  Chat  is.  writers  get  too  engaged  in  making  their 
iK  ament  look  pretty .  Not  only  that,  many  writers  arc  bad  at  it.  I  heir  documents  are  uglier  and  harder 
•o  underhand  than  documents  dial  are  professionally  typeset.  The  complaint  goes  on  to  suggest  that 
,x • : t  -r-.  -should  either  be  trained  in  how  format  text  quickly  and  effective!.,  or  they  should  leave  text 
!,  m-  g  u>  those  who  are  trained  at  it.  Superficially,  there  seems  to  be  a  analogy  hero  to  NotcCards. 
a  m  h  ..  n  (.enrages  thinkers  to  explore  organizations  of  their  arguments.  I  he  analogy  says  that  fooling 
.  s  a.".:  invent  organizations  will  just  waste  die  thinker's  time.  Someone  el-e  should  do  it.  However, 
a.-.  :■  n.>  L.tdre  of  profe'Mon.sl  argument-organizers  who  arc  the  analogs  of  printers  and  graphics 
tz-c'v  N  i  >  r  i .  i  * :  ’beiv  he  Mich  a  cadre.  I  lie  organization  of  arguments  is  too  strongly  coupled  to  their 
'  'r.  ■nti  :'.'  the  torm.it  of  text  is  weakly  coupled,  so  professional  text  formatters  can  and  do 

-•  -  V:\  vl.tte reni! . .  ai cement  :imn  has  an  organization  whether  one  likes  it  or  not.  (I  ikexvisc. 
"  i;.  like  it  <>;  r’ot.i  Someone  has  to  give  the  arguments  a  good  organization.  And  that 


someone  must  he  tlie  dimker  who  is  responsible  t'oi  the  arguments  content.  In  short,  exploring 
argument  organizations  is  a  necessary  activity.  not  a  waste  of  the  dnnker's  time. 

Notecarjs  encourages  a  particular  kind  of  exploration.  To  see  this,  let's  compare  the  process  of 
organizing  arguments  using  NoteCards  with  the  same  process  using  paper  note  cards.  With  paper  note 
cards,  a  spatial  organization  is  external  to  the  content  of  the  note  cards.  You  don't  have  to  change  what 
the  card  says  in  order  to  move  it  from  one  tile  to  another.  On  the  other  hand  NoteCards'  browser  is 
driven  by  the  content  of  the  cards,  namely  the  pointers  that  are  part  of  their  texts.  To  reorganize  a 
browser,  you  must  change  the  cards'  content.  To  put  it  differently.  NoteCards'  organizations  arc 
! vticrgoii  properties  of  the  content  of  the  argument s:  they  are  not  imposed  from  the  outside.  In 
retrospect,  both  the  tree  organization  dial  I  used  for  my  arguments  in  the  document  and  the  list-likc 
chapter  structure  were  external  organizations.  They  were  so  decoupled  from  the  content  of  die 
arguments  that  1  was  able  to  fool  myself  (and  the  readers!)  into  believing  that  more  was  shown  than 
actually  was  shown.  Notecards'  emphasis  on  emergent  organizations  makes  it  harder  to  fool  oneself. 

Perhaps  the  hardest  job  that  a  theorist  has  is  to  discover  the  assumptions  that  he  or  she  is  making. 
Goodman  (1955)  has  pointed  out.  in  connection  with  his  famous  Grue-BIccn  example,  diat  important 
scientific  assumptions  may  hide  in  the  very  vocabulary  one  uses  to  think  about  the  dicory.  Uncovering 
such  assumptions  is  so  difficult  that  any  aid.  even  indirect  aid.  would  be  welcome.  NoteCards  seems  to 
provide  such  aid.  At  least,  it  helped  me  uncover  a  class  of  assumptions,  namely  the  decomposition 
issues,  that  lay  deeply  buried  in  the  technical  vocabulary  of  computer  science  and  mathematics.  lTie 
discovery  of  die  annotated  grammars  decomposition  is  a  case  in  point. 

What  happens  to  a  N'otcCards  database  eventually?  Mine  is  too  big  to  publish,  even  as  a  book.  One 
wonders  if  it  is  worth  the  effort  to  type  in  all  those  NotcCards  when  all  one  can  hope  to  gain  is  a  better 
theory  (as  if  that  weren't  enough!).  There  arc  several  answers,  ranging  from  mundane  to  futuristic, 
f  irst,  it  isn't  all  diat  much  typing.  It  took  only  ten  days  to  get  my  database  started.  Second.  NotcCards 
has  facilities  for  stringing  together  cards  into  a  single  document.  Thus,  with  a  little  extra  effort,  one  can 
obtain  a  rough  draft  of  a  paper,  or  perhaps  several  papers. 

On  a  more  personal  note.  I  find  that  NotcCards  falls  into  a  natural  nitch  that  is  midway  between 
publications  and  lab  notes.  Publications  have  to  be  carefully  worked  over  to  be  both  an  accurate, 
consistent  presentation  of  the  dicory  ami  an  understandable  one.  On  the  other  hand,  lab  notes  arc 
private  memoranda,  where  ideas  arc  worked  out  in  rough.  One's  recent  lab  notes  arc  often  based  on 
assumptions  that  contradict  earlier  lab  notes,  reflecting  the  process  of  theory  revision.  As  a  whole,  lab 
notes  are  inconsistent  and  sometimes  vague,  and  lack  the  exposition  al  "sugar"  of  publications.  The 
NotcCards  database  is  an  accurate,  consistent  present. ition  of  the  thcor, .  but  a  private  one  that  need  not 
have  expositional  sugar.  Metaphorically  speaking,  it  represents  die  integral  over  time  of  die  lab  notes. 
Ml  the  contradictions,  reversals,  and  dead  ends  arc  removed. 

I  o  see  why  tins  is  important,  consider  an  analogy.  Suppose  a  compiitei  scientist  builds  a  program  and  a 
mathematician  builds  a  proof.  Ilotli  the  program  and  the  proof 'can  be  accurately  represented  in  writing. 
However  the  typical  publication  doesn't  print  the  program  or  proof,  but  only  the  key  ideas  and  a 
discussion.  Nonetheless,  constructing  the  program  or  proof  is  a  meihodulogic.il  necessity  because  it 


allows  one  to  clarify  the  ideas  .md  make  them  i  igorous.  On  a  deeper  lo\ el.  it  may  be  important  to  have  a 
^i  nuvte  embodiment  (i.e..  a  written  one)  of  the  program  or  proof  as  an  aid  to  thinking  about  it.  Now 
foi  the  analogs.  \  theorist  builds  a  theory,  and  die  publications  discuss  it.  just  as  a  computer  scientist 
build'  a  program  and  die  publications  discuss  it.  or  the  mathematician  builds  a  proof  and  the 
publications  discuss  it.  Hut  unlike  programs  or  proofs,  a  theory  usually  has  no  accurate,  written 
representation.  I  hat  is.  until  NoteCards  and  its  ilk  were  developed.  I  he  NoteCards  database  is  about  as 
close  as  any  written  artifact  can  get  to  expressing  a  whole  theory.  If  the  analogy  holds,  then  we  can 
exf>c\t  NoteCards  to  help  theorists  clarify  their  ideas  and  make  them  rigorous.  Moreover,  die 
NoteCards  database  should  serve  as  a  concrete  embodiment  of  die  theorv.  I  Ins  last  feature  seems 
t'u.'ficulurlv  important  to  me.  Nowadays.  I  view  my  work  as  building  a  NoteCards  database  qua  theory, 
io  tbeot i/e  without  building  a  NoteCards  database  seems  like  programming  without  building  a 
program.  One  can  do  it.  hut  it's  harder. 

because  NoteCards  databases  are  accurate  representations  of  theories,  diev  have  excellent  potential  as 
chicles  for  collaboration.  Moreover,  because  die  user  interface  lies  somewhere  between  a  blackboard 
and  a  text  editor.  NoteCards  may  augment  or  replace  them  as  die  customary  focus  of  a  collaborative 
theory -development  effort.  It  is  easy  to  imagine  two  people  in  different  cities  both  editing  die  same 
NoteCards  screen  while  the  talking  over  die  phone. 

being  halfway  between  lab  notes  and  journal  articles  may  also  make  NoteCards  a  unique  aid  to 
c'aduatc-ievel  teaching.  A  Notecard  database  would  allow  young  dicorists  to  crawl  around  inside  a 
, !  :>,te  theory,  getting  to  understand  it  more  deeply  than  they  could  from  journal  articles.  Incidentally, 
mis 's  o  ne  answer  to  what  could  happen  to  NoteCard  databases  after  their  active  development  ceases 
i  r.ev  might  rest  in  graduate  schools,  embalmed  in  computational  display  cases  for  students  to  dissect. 

!  lie  :  ist  comment  should  be  that  NoteCards  is  only  a  beginning.  One  can  imagine  many  facilities  that 
•.eeld  he  .id. led  to  it  to  make  theory  development  even  better.  For  instance,  a  truth  maintenance  system 
ide  kieer  in  preparation;  Doyle,  1979:  McDermott.  1983)  would  make  it  easier  to  revise  the  theory. 
’A  hen  a  hypothesis  changes  from  winning  to  losing,  the  IMS  would  automatically  retract  all  the 
arguments  that  depend  on  it.  Thai  may.  in  turn,  cause  other  hypotheses  to  lose  their  support  and 
he.  ome  losers.  Currently,  such  propagation  is  done  by  hand.  Adding  a  I  MS  to  NoteCards  is  just  one 
:■  no  pic  of  how  automatic  infercncing  techniques  could  be  imported  from  Artificial  Intelligence. 
N  :c<  an K  is  a  first  step  on  the  path  towards  a  theoretician's  workbench  that  is  a  synergistic  combination 
'  hear  in  and  artificial  intelligence. 
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I.  An  issue:  something  that  the  theory  must  take  a  stand  on 

A.  Hypothesis  1 

1.  Pro 

a.  an  argument  in  favor  of  the  hypothesis  1 

i.  an  empirical  fact  used  in  the  argument 

ii.  another  fact... 

b.  another  argument  in  favor  of  hypothesis  1 

i.  an  empirical  fact... 

2.  Con 

a.  an  argument  against  hypothesis  1 

i.  an  empirical  fact  used  in  die  argument 

ii.  another  fact... 

b.  another  argument  against  hypothesis  1 

c.  another... 

3.  Moot 

a.  an  argument  that  bears  on  hypothesis  1,  but  is  not  decisive 

b.  another  such  argument 

B.  Hypothesis  2 

1.  Pro 

a.  an  argument  in  favor  of  die  hypothesis  2 

b.  another  argument  in  fa\  or  of  die  hypothesis  2 

2.  Con 

a.  an  argument  against  hypothesis  2 

b.  another... 

3.  Moot 

a.  an  argument  that  bears  on  the  hypodicsis  2,  but  is  not  decisive 

b.  another  such  argument 

II.  Another  issue.... 

Figure  2 

An  argument  tree,  displayed  as  an  outline. 

There  is  one  notccard  for  each  issue,  one  for  each  hypothesis,  one  for  each  argument,  and  one  for  each 
empirical  fact.  I  he  issue  notccard  points  to  the  relevant  hypothesis  notecards  (c.g.,  for  die  above  tree, 
card  1  would  point  to  cards  A  and  B).  Hach  hypothesis  notccard  points  to  the  argument  notecards  that 
concern  it.  An  argument  notccard  points  to  die  facts  that  it  references. 


MattA*  format  for  arguments 


1  I  lscelaneous  comments.  it  any.  lie  re. 

The  above  is  a*4  by-:i  rn at tri::.  The  columns  ar-  labelled  by 
pointer.'  to  pieces  of  evidence;  the  ro!,fs  are  la),  -  lied  by  pointer.' 
to  hypotheses.  The  cells  ot  rhe  mat  tri”  contain  pointers  to 
arguments  If  a  hypothesis  can  explain  a  piece  of  evidence,  the 
corresponding  pointer  is  labelled  "  +  A  -  Libel  means  that 
the  evidence  contradicts  the  hypothesis.  A  "U"  label  means  the 
evidence  neither  supports  nor  contradicts  the  hyp o thesis. 


Figure  3 

A  Cartesian  product  notecard 
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